Method and system for far field image absolute navigation sensing

ABSTRACT

A method and system for far field image navigation sensing are described. The method includes pre-processing an image from a remote beacon into gray scale and contour data. Multiple frames of the image, including the gray scale and contour data are buffered. The beacon image is reconstructed from the buffered frames, which include blur data related to the beacon image. A position of the beacon is measured from the buffered frames. The blur data are analyzed to generate a blur vector. The beacon position and blur vector are processed to generate navigation information based on the beacon.

TECHNOLOGY

The present invention relates generally to the field of electronics.More specifically, embodiments of the present invention relate to imagenavigation sensing.

BACKGROUND

Electronic media such as television (TV) and computerized apparatus havemany modern roles. Education and entertainment are among the mostfamiliar of these roles. TV displayed electronic games have become verypopular and commercially successful exemplars of these media and can bequite useful in fulfilling these roles.

To take advantage of contrasts in lighting sources, some TV basedelectronic games are used in a darkened or partially lit environment. Agame player for example may sit comfortably on a couch, chair, cushionor carpet and use a remote control device to interact with an electronicgame that is displayed to them on their TV from across the darkenedroom.

Some remote control gaming interfaces include a camera device, whichsenses the relative position of the remote control unit with respect tothe TV screen. This provides a relative position input to a processorthat controls the game. In such game modalities, the relative brightnessof the TV screen provides an effective navigational beacon to achievethis input.

If the game player (e.g., user) moves the remote control unit, itscamera “sees” the images upon which it is trained seem to move, thussensing apparent motion from the TV screen beacon. The game controllertranslates related input from the camera device in terms of pixels andunits of time. However, rapid movement of the remote control unit by thegame player can be somewhat problematic. With rapid movement of theremote control device, calculation with the vector information becomesdifficult to achieve between successive frames.

Thus, successive correlation algorithms and related processes used bythe game controller can experience difficulty in achieving high speedabsolute position sensing in free space using the TV screen as itsbeacon. Synchronization may be lost because, even where the remotecontrol unit is returned to its original position, errors in thecalculation process prevent the console from returning the TV screenimage to the center of the console. This will typically be contrary tothe user's expectation and can adversely affect the user's gamingexperience.

SUMMARY

A method and system for far field image navigation sensing aredisclosed. The method comprises pre-processing an image from a remotebeacon into gray scale and contour data. Multiple frames of the image,including the gray scale and contour data are buffered. The beacon imageis reconstructed from the buffered frames, which comprise blur datarelated to the beacon image. A position of the beacon is measured fromthe buffered frames. The blur data are analyzed to generate a blurvector. The beacon position and blur vector are processed to generatenavigation information based on the beacon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a flowchart of an exemplary method for far field imagenavigation sensing, according to an embodiment of the present invention.

FIG. 2 depicts an exemplary computer based system for far field imagenavigation sensing, according to an embodiment of the present invention.

FIG. 3 depicts another exemplary system for far field image navigationsensing, according to an embodiment of the present invention.

DETAILED DESCRIPTION

Exemplary embodiments of methods and systems for far field imagenavigation sensing are described below. Reference will now be made indetail to embodiments of the present invention, examples of which areillustrated in the accompanying drawings. While the present inventionwill be described in conjunction with the following embodiments, it willbe understood that they are not intended to limit the present inventionto these embodiments alone. On the contrary, the present invention isintended to cover alternatives, modifications, and equivalents which maybe included within the spirit and scope of the present invention asdefined by the appended claims.

Furthermore, in the following detailed description of exemplaryembodiments of the present invention, numerous specific details are setforth in order to provide a thorough understanding of the presentinvention. However, one of ordinary skill in the art will realize thatembodiments of the present invention may be practiced without thesespecific details. In other instances, well-known devices, methods,systems, processes, procedures, components, circuits and apparatus,protocols, standards, etc. have not been described in detail so as notto unnecessarily obscure aspects of the present invention.

Portions of the detailed description that follows are presented anddiscussed in terms of a process. Although blocks and sequencing thereofare disclosed in a flowchart figure herein (e.g., FIG. 1) describing theoperations of this process, (e.g., process 10), such blocks andsequencing are exemplary. Embodiments of the present invention are wellsuited to performing various other blocks or variations of the blocksrecited in the flowchart of the figure herein, and in a sequence, order,etc. other than that depicted and described herein, except as describedherein otherwise.

In one embodiment, a process for far field image navigation sensing isperformed with a computer based system. Means for performing a processfor far field image navigation sensing include, in various embodiments,a variety of computer and imaging systems, devices, apparatus andcommunication media, including but not limited to cameras or other imagecapture devices, televisions and/or computer monitors, wireless and/orwire-based media and include software, hardware, firmware and/orcombinations thereof. In one embodiment, such a process is performedwith a computer under the control of computer readable code encoded upon(e.g., within) a computer readable medium.

Embodiments of the present invention relate to methods and systems forfar field image navigation sensing. In one embodiment, a method for farfield image navigation sensing comprises pre-processing an image from aremote beacon into gray scale and contour data. Multiple frames of theimage, including the gray scale and contour data are buffered. Thebeacon image is reconstructed from the buffered frames, which compriseblur data related to the beacon image. A position of the beacon ismeasured from the buffered frames. The blur data are analyzed togenerate a blur vector. The beacon position and blur vector areprocessed to generate navigation information based on the beacon.

Therefore, navigation sensing with vector information is achievedbetween successive frames even with rapid movement of remote controlunits, e.g., by a game player, which can be somewhat problematic withconventional approaches. Game controllers can overcome conventionaldifficulties to achieve high speed absolute position sensing in freespace, e.g., using a TV screen as a beacon. Synchronization is preservedwhere remote control units are returned to an original position becauseerrors in the calculation process are reduced, which conventionallyprevent the console from returning the TV screen image to the center ofthe console. Thus, embodiments of the present invention conform totypical user expectations and promote the user's gaming experience.

Exemplary Method

FIG. 1 depicts a flowchart of an exemplary method 10 for far field imagenavigation sensing, according to an embodiment of the present invention.Method 10 begins with block 11, wherein an image from a remote beacon ispre-processing into gray scale and contour data.

In one embodiment, the remote beacon comprises a television (TV) screen.In one embodiment, the remote beacon comprises a computer monitor. Inone embodiment, an image of the remote beacon is captured by a highspeed camera or similar image capture device.

In block 12, multiple frames of the image, including the gray scale andcontour data, are buffered. In block 13, the beacon image isreconstructed from the buffered frames, which comprise blur data relatedto the beacon image.

In block 14, a position of the beacon is measured from the bufferedframes. In block 15, the blur data are analyzed to generate a blurvector. In block 16, the beacon position and blur vector are processedto generate navigation information based on the beacon.

In optional block 17, the navigational information is transmitted to aremote device. Transmission in various embodiments takes place overwireless and/or wire based media. In optional block 18, a controlfunction is performed that is based on the navigational information. Thecontrol function can be applied to control of an electronic game,displayed for instance on the remote monitor.

Exemplary Systems

FIG. 2 depicts an exemplary computer based system 200 for far fieldimage navigation sensing, according to an embodiment of the presentinvention. An image from a remote beacon such as a TV screen or computermonitor is captured by an image capturer 202, which can comprise a highspeed camera or a similar image capture device. Image capturer 202provides an input of the beacon image to system 200.

System 200 includes a gray scale and contour processor 203, whichreceives the beacon image. Grey scale and contour processor 203processes the remote beacon 201 image input from image capturer 202 intogray scale and contour data. Multiple frames of the image, including thegray scale and contour data associated therewith, are buffered withframe buffer 204.

Image reconstructor 205 reconstructs the beacon image from the bufferedframes, which include blur data. Beacon position measurement module 206measures the position of the beacon from the gray scale and contour datain the buffered frames. Blur analyzer 207 analyzes the blur data thereinand generates a corresponding blur vector. In one embodiment, beaconposition measurement module 206 and blur analyzer 207 comprise a part,function, etc. of image reconstructor 205.

Processor 208 processes the beacon position measurement and blur vectorto generate navigation information based on the remote beacon 201 andcontrols gray scale/contour processor 203 therewith.

In one embodiment, the navigational information generated with processor208 is transmitted with transmitter 209, via a medium 212, to receiver210. In one embodiment, medium 212 comprises a wireless medium. In oneembodiment, medium 212 comprises a wire based medium. The navigationalinformation received with receiver 210 is supplied to a controller 211,which performs a control function based thereon. For instance,controller 211 can control a game displayed on remote beacon 201 withthe navigational information.

FIG. 3 depicts another exemplary system 300 for far field imagenavigation sensing, according to an embodiment of the present invention.System 300 effectuates high speed absolute three axis tracking, using ahigh frame rate camera (or another image capture device) with a multiplenavigation engine 325.

Navigation engine 325 functions with frame buffer 304 to reconstruct TVscreen (or computer monitor, etc.) 301 as a navigation beacon.Advantageously, system 300 also uses information derived from theblurred beacon image for navigation functions. System 300 achieves highspeed, effectively absolute position sensing in free space, using TV 301as a navigational beacon. System 300 uses a fraction of the image thatis displayed on TV 301.

Camera 300 operates at a high frame rate. In one implementation, camera302 operates at 120 frames per second (fps). In one embodiment, camera302 looks at the scanning electronic beam (e-beam) of TV 301. Image datacaptured with camera 302 is pre-processed with first processor 303 intogray scale and contour data, e.g., with methods known in the art. In oneimplementation an eight level gray scale is used.

Frame buffer 304 stores multiple frames from the high speed camera 302.The image displaying on TV beacon 301 can comprise interlaced frameimages at a frame rate somewhat slower than that at which camera 302operates. In one implementation, TV 301 operates at a frame rate of 30fps. Thus, in one implementation, buffer 304 stores the four most recentframes. The multiple frames are combined, e.g., with image addingtechniques known in the art, to achieve an effective reconstruction ofthe image displaying on TV 301, e.g., substantially in real time or innear real time.

The reconstructed image may show some deformation because camera 302 maybe deployed with a remote control unit, which may be moved by a user.Importantly however, the images captured with camera 302 from TV beacon301 are effectively reconstructed in the memory of the navigationalsystem 300, e.g., with buffer 304. Thus, any such distortion is usedtherewith as a source of information relating to direction and speed ofmotion, e.g., of the user held remote control device with which camera302 is deployed.

The reconstructed image comprises an input to navigation engine 325. Inone embodiment, navigation engine 325 comprises a beacon (e.g., TV)position determining entity 305 and a blur analyzer 306. Positiondeterminer 305 effectively measures the absolute position (x, y) of TV301 within the field of view of camera 302.

The reconstructed image input from frame buffer 304 comprises asignificant quantity of blur data, e.g., related to the movement of aremote control unit with which camera 302 is deployed. This blur data isanalyzed with blur analyzer 306, which generates a corresponding blurvector (dx, dy).

The absolute position (x, y) of TV 301 and the corresponding blur vector(dx, dy) comprise inputs to second processor 307. In one embodiment,processor 307 functions with storage 308. In one embodiment, storage 308comprises a memory device. In one embodiment, memory device 308comprises random access memory (RAM).

Processor 307 processes the absolute position data (x, y) and the blurvector (dx, dy) and generates navigational information, based on the TVbeacon 301. The navigational information provides in one embodiment topre-processor 303. Further, in one embodiment, the navigationalinformation generated with processor 307 provides a useful navigationbased control signal.

For instance, in one implementation, navigation based informationgenerated with processor 307 is transmitted with transmitter 309 viawire based and/or wireless interface and media 311 to a receiver 310.Receiver 310 provides the navigational information to a control entity312.

In one embodiment, control entity 312 comprises a controller, such asfor an electronic game. Images related to the electronic game aredisplayed on the TV 301. Thus, the navigation based control signalcontrols the image displayed on TV beacon 301.

In one sense, embodiments of the present invention advantageouslysimulate humanlike brain-eye action. The relatively fast acting humaneye for instance sees the raster scan of TV 301. The human brain howeverfunctions relatively more slowly in the visual sense, ignoring theraster scan and “seeing,” in the mental sense, the image beingelectronically “painted” with the raster scan.

System 300 is similarly dualistic: high speed camera 302 “sees” thee-beam raster scan in a way analogous to a humanlike eye. The buffer 304and processor 307, in reconstructing the image from the four most recentframes and performing navigation related calculations with it,co-function analogously to a humanlike brain.

Embodiments of the present invention, methods and systems for far fieldimage navigation sensing, are thus described. While the presentinvention has been described in particular embodiments, it should beappreciated that the present invention should not be construed aslimited by such embodiments, but rather construed according to thefollowing claims.

1. A method for far field image navigation sensing, comprising: capturing an image displayed on a display screen of a remote beacon using an image capture device; pre-processing said image displayed on said display screen into gray scale and contour data; buffering a plurality of frames of said image displayed on said display screen wherein said frames comprise said gray scale and contour data; reconstructing said image displayed on said display screen from said buffered frames wherein said buffered frames comprise blur data related to said image; measuring a position of said beacon from said buffered frames; analyzing said blur data to generate a blur vector; and processing said beacon position and said blur vector to generate navigation information based on said beacon.
 2. The method as recited in claim 1 wherein said remote beacon comprises at least one of a television screen and a computer monitor.
 3. The method as recited in claim 1, further comprising: performing a control function based on said navigation information.
 4. The method as recited in claim 1, further comprising: transmitting said navigation information to a remote device.
 5. The method as recited in claim 4 wherein said remote device comprises a control device.
 6. The method as recited in claim 5 wherein said remote device comprises a game controller.
 7. The method as recited in claim 6 wherein game controller controls a game wherein said game is displayed on said remote beacon.
 8. A system for far field image navigation sensing, comprising: an image capture device, for sensing image information displayed on a display screen of a remote beacon; a first processor coupled to said image capture device, for pre-processing said image information displayed on said display screen into gray scale and contour data; a buffer coupled to said first processor, for storing a plurality of frames of said image information displayed on said display screen wherein said frames comprise said gray scale and contour data; a beacon position analyzer coupled to said buffer, for measuring a position of said beacon from said gray scale and contour data; a blur analyzer coupled to said buffer, for generating a blur vector based on said buffered frames; and a second processor coupled to said beacon position analyzer and said blur analyzer, for processing said beacon position and said blur vector to generate navigation information based on said beacon.
 9. The system as recited in claim 8 wherein said remote beacon comprises at least one of a television screen and a computer monitor.
 10. The system as recited in claim 1, further comprising: a storage component coupled to said second processor, for storing data related to said processing.
 11. The system as recited in claim 10 wherein said storage component comprises memory.
 12. The system as recited in claim 11 wherein said memory comprises random access memory.
 13. The system as recited in claim 8, further comprising: a transmitting component coupled to said second processor, for transmitting said navigation information; a receiving component coupled to said transmitting component, for receiving said navigation information therefrom; and an interface medium for coupling said receiving component to said transmitting component.
 14. The system as recited in claim 13 wherein said interface medium comprises a wire-based medium.
 15. The system as recited in claim 13 wherein said interface medium comprises a wireless medium.
 16. The system as recited in claim 13, further comprising: a control device coupled to said receiving component, for performing a control function based on said navigation information.
 17. The system as recited in claim 16 wherein said control device comprises a game controller.
 18. The system as recited in claim 17 wherein game controller controls a game wherein said game is displayed on said remote beacon.
 19. A non-transitory computer readable medium having encoded thereon computer usable code for controlling a computer system to perform a process for far field image navigation sensing, said process comprising: pre-processing an image displayed on a display screen of a remote beacon into gray scale and contour data, wherein said image displayed on said display screen is captured using an image capture device; buffering a plurality of frames of said image displayed on said display screen wherein said frames comprise said gray scale and contour data; reconstructing said image displayed on said display screen from said buffered frames wherein said buffered frames comprise blur data related to said image; measuring a position of said beacon from said buffered frames; analyzing said blur data to generate a blur vector; and processing said beacon position and said blur vector to generate navigation information based on said beacon.
 20. A system for far field image navigation sensing, comprising: an image capture device to capture an image displayed on a display screen of a remote beacon; a first processor configured to process said image displayed on said display screen into gray scale and contour data; means for buffering a plurality of frames of said image displayed on said display screen wherein said frames comprise said gray scale and contour data; means for reconstructing said image displayed on said display screen from said buffered frames wherein said buffered frames comprise blur data related to said image; means for measuring a position of said beacon from said buffered frames; means for analyzing said blur data to generate a blur vector; and a second processor configured to process said beacon position and said blur vector to generate navigation information based on said beacon. 